Some image-based application (or software) automation systems include a record phase and a playback phase. During the record phase, an image-based application automation system records actions for an application that will be automated during a playback phase. For example, an image-based application automation system detects input events at a graphical user interface and captures an image of the graphical user interface in response to each input event. Often, the image-based application automation system then defines an automation action for each input event that specifies the image for that input event, and automation actions defined during the record phase are stored within a script file.
During the playback phase, the image-based application automation system performs the actions recorded during the record phase. For example, the image-based application automation system can identify the automation actions in the script file and replay the input events specified in the automation actions. That is, for each automation action, the image-based application automation system identifies a portion of a graphical user interface of the application that matches the image specified in that automation action and performs the input event at that portion of the application.